Papers by Md Nishat Raihan
MojoBench: Language Modeling and Benchmarks for Mojo (2025.findings-naacl)
Copied to clipboard
| Challenge: | Mojo is a programming language that has been praised for its speed and performance over Python. |
| Approach: | They propose a framework for Mojo code generation that evaluates code Large Language Models (LLMs) they propose 'mojo-Coder' which is the first LLM pretrained and fine-tuned for MoJO code generation . |
| Outcome: | MojoBench is the first framework for mojo code generation . it achieves a 30-35% performance improvement over leading models like GPT-4o and Claude-3.5-Sonnet . |
mHumanEval - A Multilingual Benchmark to Evaluate Large Language Models for Code Generation (2025.naacl-long)
Copied to clipboard
| Challenge: | Current evaluations focus on English-to-Python conversion tasks with limited test cases . code generation from low-resource language prompts remains largely unexplored . |
| Approach: | They propose a benchmark that supports prompts in over 200 natural languages . they provide expert human translations for 15 diverse natural languages (NLs) |
| Outcome: | The HumanEval Benchmark is the most widely used code generation benchmark . it provides expert human translations for 15 diverse natural languages . |